Many enterprise problems use data collected over time. In some cases, the data is collected frequently and over a long period of time, resulting in many time-stamped observations. In situations such as in financial settings and/or in the telecommunications industry and/or in academia and so on, it is convenient to use an empirically-collected set of time-stamped data to derive a model, which model uses a smoothing function to represent the empirically-collected data. The derived model and smoothing function are used to forecast future observations.
Many model generation implementations are found in enterprise settings, including in statistical packages such as “JMP”, “SPSS”, and “R”, however legacy implementations are deficient in at least three aspects. First, legacy implementations of time-series smoothing fail for large datasets (e.g., they run out of memory). Second, legacy implementations are either uncoupled or only loosely coupled to a database engine that is capable of storing large datasets. In addition, legacy implementations rely on algorithms that include a dependency on the calculation of a state in a previous time step when updating the state of the current time step. This step-to-step dependency found in legacy approaches prevents concurrent calculations of states in such legacy implementations.
None of the aforementioned legacy approaches achieve the capabilities of the herein-disclosed techniques for exploiting parallelism during smoothing of large-scale discrete datasets to build a predictive model. Therefore, there is a need for improvements.